Tutorial on removing the shackles of regression analysis: How to stay true to your theory of binary response probabilities.

Statistical analyses of data often add some additional constraints to a theory and leave out others, so as to convert the theory into a testable hypothesis. In the case of binary data, such as yes/no responses, or such as the presence/absence of a symptom or a behavior, theories often actually predict that certain response probabilities change monotonically in a specific direction and/or that certain response probabilities are bounded from above or below in specific ways. A regression analysis is not really true to such a theory in that it may leave out parsimonious constraints and in that extraneous assumptions like linearity or log-linearity, or even the assumption of a functional relationship, are dictated by the method rather than the theory. That mismatch may well bias the results of empirical analysis and jeopardize attempts at meaningful replication of psychological research. This tutorial shows how contemporary order-constrained methods can shed more light on such questions, using far weaker auxiliary assumptions, while also formulating more detailed, nuanced, and concise hypotheses, and allowing for quantitative model selection. (PsycINFO Database Record (c) 2019 APA, all rights reserved).

Is Cognitive Impairment Related to Violations of Rationality? A Laboratory Alcohol Intoxication Study Testing Transitivity of Preference.

Alcohol intoxication is well known to impair a number of cognitive abilities required for sound decision making. We tested whether an intoxicating dose of alcohol altered whether individuals satisfied a basic property of rational decision making, transitivity of preference. Our study was within-subjects in design and our analysis teased apart stable, yet error-prone, preferences from variable, error-free preferences. We find that alcohol intoxication does not appear to play a major role in determining whether subjects violate transitivity. For a minority of individuals, we find that alcohol intoxication does impact how they select among and/or perceive lotteries with similar attribute values. This, in turn, can cause them to alter various aspects of their preference structure.

Short abstinence from online social networking sites reduces perceived stress, especially in excessive users.

Online social networking sites (SNSs), such as Facebook, provide frequent and copious social reinforcers (e.g., "likes") delivered at variable time intervals. As a result, some SNS users display excessive, maladaptive behaviors on these platforms. Excessive SNS users, and typical users alike, are often aware of their intense use and psychological dependence on these sites, which may lead to elevated stress. In fact, research has demonstrated that use of SNSs alone induces elevated stress. Other research has begun to investigate the effects of short periods of SNS abstinence, revealing beneficial effects on subjective wellbeing. We aligned these two lines of research and hypothesized that a short period of SNS abstinence would induce a reduction in perceived stress, especially in excessive users. The results confirmed our hypothesis and revealed that both typical and excessive SNS users experienced reduction in perceived stress following SNS abstinence of several days. The effects were particularly pronounced in excessive SNS users. The reduction in stress was not associated with academic performance increases. These results indicate a benefit-at least temporarily-of abstinence from SNSs and provide important information for therapists treating patients who struggle with excessive SNS use.

A model-based test for treatment effects with probabilistic classifications.

Within modern psychology, computational and statistical models play an important role in describing a wide variety of human behavior. Model selection analyses are typically used to classify individuals according to the model(s) that best describe their behavior. These classifications are inherently probabilistic, which presents challenges for performing group-level analyses, such as quantifying the effect of an experimental manipulation. We answer this challenge by presenting a method for quantifying treatment effects in terms of distributional changes in model-based (i.e., probabilistic) classifications across treatment conditions. The method uses hierarchical Bayesian mixture modeling to incorporate classification uncertainty at the individual level into the test for a treatment effect at the group level. We illustrate the method with several worked examples, including a reanalysis of the data from Kellen, Mata, and Davis-Stober (2017), and analyze its performance more generally through simulation studies. Our simulations show that the method is both more powerful and less prone to type-1 errors than Fisher's exact test when classifications are uncertain. In the special case where classifications are deterministic, we find a near-perfect power-law relationship between the Bayes factor, derived from our method, and the p value obtained from Fisher's exact test. We provide code in an online supplement that allows researchers to apply the method to their own data. (PsycINFO Database Record (c) 2018 APA, all rights reserved).

A model-based analysis of decision making under risk in obsessive-compulsive and hoarding disorders.

Attitudes towards risk are highly consequential in clinical disorders thought to be prone to "risky behavior", such as substance dependence, as well as those commonly associated with excessive risk aversion, such as obsessive-compulsive disorder (OCD) and hoarding disorder (HD). Moreover, it has recently been suggested that attitudes towards risk may serve as a behavioral biomarker for OCD. We investigated the risk preferences of participants with OCD and HD using a novel adaptive task and a quantitative model from behavioral economics that decomposes risk preferences into outcome sensitivity and probability sensitivity. Contrary to expectation, compared to healthy controls, participants with OCD and HD exhibited less outcome sensitivity, implying less risk aversion in the standard economic framework. In addition, risk attitudes were strongly correlated with depression, hoarding, and compulsion scores, while compulsion (hoarding) scores were associated with more (less) "rational" risk preferences. These results demonstrate how fundamental attitudes towards risk relate to specific psychopathology and thereby contribute to our understanding of the cognitive manifestations of mental disorders. In addition, our findings indicate that the conclusion made in recent work that decision making under risk is unaltered in OCD is premature.

On the Functional Form of Temporal Discounting: An Optimized Adaptive Test.

The tendency to discount the value of future rewards has become one of the best-studied constructs in the behavioral sciences. Although hyperbolic discounting remains the dominant quantitative characterization of this phenomenon, a variety of models have been proposed and consensus around the one that most accurately describes behavior has been elusive. To help bring some clarity to this issue, we propose an Adaptive Design Optimization (ADO) method for fitting and comparing models of temporal discounting. We then conduct an ADO experiment aimed at discriminating among six popular models of temporal discounting. Rather than supporting a single underlying model, our results show that each model is inadequate in some way to describe the full range of behavior exhibited across subjects. The precision of results provided by ADO further identify specific properties of models, such as accommodating both increasing and decreasing impatience, that are mandatory to describe temporal discounting broadly.

A Tutorial on Adaptive Design Optimization.

Experimentation is ubiquitous in the field of psychology and fundamental to the advancement of its science, and one of the biggest challenges for researchers is designing experiments that can conclusively discriminate the theoretical hypotheses or models under investigation. The recognition of this challenge has led to the development of sophisticated statistical methods that aid in the design of experiments and that are within the reach of everyday experimental scientists. This tutorial paper introduces the reader to an implementable experimentation methodology, dubbed Adaptive Design Optimization, that can help scientists to conduct "smart" experiments that are maximally informative and highly efficient, which in turn should accelerate scientific discovery in psychology and beyond.

Discriminating Among Probability Weighting Functions Using Adaptive Design Optimization.

Probability weighting functions relate objective probabilities and their subjective weights, and play a central role in modeling choices under risk within cumulative prospect theory. While several different parametric forms have been proposed, their qualitative similarities make it challenging to discriminate among them empirically. In this paper, we use both simulation and choice experiments to investigate the extent to which different parametric forms of the probability weighting function can be discriminated using adaptive design optimization, a computer-based methodology that identifies and exploits model differences for the purpose of model discrimination. The simulation experiments show that the correct (data-generating) form can be conclusively discriminated from its competitors. The results of an empirical experiment reveal heterogeneity between participants in terms of the functional form, with two models (Prelec-2, Linear in Log Odds) emerging as the most common best-fitting models. The findings shed light on assumptions underlying these models.

Optimal Decision Stimuli for Risky Choice Experiments: An Adaptive Approach.

Collecting data to discriminate between models of risky choice requires careful selection of decision stimuli. Models of decision making aim to predict decisions across a wide range of possible stimuli, but practical limitations force experimenters to select only a handful of them for actual testing. Some stimuli are more diagnostic between models than others, so the choice of stimuli is critical. This paper provides the theoretical background and a methodological framework for adaptive selection of optimal stimuli for discriminating among models of risky choice. The approach, called Adaptive Design Optimization (ADO), adapts the stimulus in each experimental trial based on the results of the preceding trials. We demonstrate the validity of the approach with simulation studies aiming to discriminate Expected Utility, Weighted Expected Utility, Original Prospect Theory, and Cumulative Prospect Theory models.

Model discrimination through adaptive experimentation.

An ideal experiment is one in which data collection is efficient and the results are maximally informative. This standard can be difficult to achieve because of uncertainties about the consequences of design decisions. We demonstrate the success of a Bayesian adaptive method (adaptive design optimization, ADO) in optimizing design decisions when comparing models of the time course of forgetting. Across a series of testing stages, ADO intelligently adapts the retention interval in order to maximally discriminate power and exponential models. Compared with two different control (non-adaptive) methods, ADO distinguishes the models decisively, with the results unambiguously favoring the power model. Analyses suggest that ADO's success is due in part to its flexibility in adjusting to individual differences. This implementation of ADO serves as an important first step in assessing its applicability and usefulness to psychology.

Adaptive design optimization: a mutual information-based approach to model discrimination in cognitive science.

Discriminating among competing statistical models is a pressing issue for many experimentalists in the field of cognitive science. Resolving this issue begins with designing maximally informative experiments. To this end, the problem to be solved in adaptive design optimization is identifying experimental designs under which one can infer the underlying model in the fewest possible steps. When the models under consideration are nonlinear, as is often the case in cognitive science, this problem can be impossible to solve analytically without simplifying assumptions. However, as we show in this letter, a full solution can be found numerically with the help of a Bayesian computational trick derived from the statistics literature, which recasts the problem as a probability density simulation in which the optimal design is the mode of the density. We use a utility function based on mutual information and give three intuitive interpretations of the utility function in terms of Bayesian posterior estimates. As a proof of concept, we offer a simple example application to an experiment on memory retention.

Behavioural social choice: a status report.

Behavioural social choice has been proposed as a social choice parallel to seminal developments in other decision sciences, such as behavioural decision theory, behavioural economics, behavioural finance and behavioural game theory. Behavioural paradigms compare how rational actors should make certain types of decisions with how real decision makers behave empirically. We highlight that important theoretical predictions in social choice theory change dramatically under even minute violations of standard assumptions. Empirical data violate those critical assumptions. We argue that the nature of preference distributions in electorates is ultimately an empirical question, which social choice theory has often neglected. We also emphasize important insights for research on decision making by individuals. When researchers aggregate individual choice behaviour in laboratory experiments to report summary statistics, they are implicitly applying social choice rules. Thus, they should be aware of the potential for aggregation paradoxes. We hypothesize that such problems may substantially mar the conclusions of a number of (sometimes seminal) papers in behavioural decision research.